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(54) Abstract Title 

Electronic zoom control in a virtual studio 



<57> A camera in a virtual studio uses a background panel 18 as a reference to allow accurate location of the 
captured image in the virtual studio image. When the camera manually zooms in on, for example a face the 
amount of the reference panel 208 seen in the field of view is reduced until at the limit of the area indicated by 
dotted line 240 the panel is no longer visible and further manual zooming is not possible without the system 
losing lock. At this point the invention allows the cameraman to select an area of the image and electronically 
zoom in on it. To increase magnification and/or image quality texture mapping may be used to implement the 
electronic zooming. 
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2324429 

VIRTUAL CA2V1ERA CONTROL SYSTEM 



The present invention relates to a video camera control system and 
more particularly to a system providing dual control of a video output 
5 from a camera. 

The system is particularly useful for a chroma-key ("blue box") and 
for a virtual set apparatus. 

10 In a virtual set apparatus a TV camera is used to capture a 

foreground object such as a presenter. In a preferred virtual set solution 
the foreground object is situated in fi-ont of a coded pattern composed of 
two shades of key colour (for example, two shades of blue). 

15 In a chroma-key apparatus the foreground object is located in a 

plain blue (or another key colour) studio and the shooting camera is 
completely stationary and cannot pan, tilt, zoom or dolly. In a first virtual 
set system as described in copending PCT Application No. WO95/30312 
to the present applicant a unique pattern is described enabling the camera 

20 to be freely moved. 

The system depends on the camera field of view being wide enough 
to include at least some of the lines in the pattern. In co-pending PCT 
Application No. GB96/02227 to the same applicant a method is described 
25 for ensuring that only a minimum number of lines of the pattern are 
required to calculate the whole camera viewpoint data ft-om the pattern. 

There may however be instances where no lines are visible because 
the camera has zoomed in too far. In such cases the system will not be 
30 able to calculate the viewpoint information and to properly render the 
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background image. To ensure that this does not occur the cameraman has 
to ensure that the field of view always includes sufficient lines of the 
background panel to maintain lock. 

This prevents very close views of either presenters or real objects 
held by presenters - such as a newspaper or book and therefore detracts 
from the system performance. 

It is an objective of the present invention to enable extremely close 
views to be obtained of foreground objects in virtual set systems without 
losing lock even if the camera continues to zoom in. It is another 
objective of the present invention to enable close ups and pan/tilt motions 
in a chroma-key production where the camera is absolutely stationary. 

The present invention provides a video control system suitable for 
a virtual studio apparatus including a video camera, said control system 
comprising electronic image control means for selecting a portion of a 
video image captured by said camera, said electronic image control means 
being operative to perform zoom/pan and or tilt operations on said 
selected portion of said video image. 

In the virtual studio apparatus the video image contains a 
foreground object and at least a portion of a patterned chroma-key panel. 

The present invention also provides a virtual studio apparatus 
comprising a patterned chroma-key panel, a TV camera situated in a 
foreground area in front of the patterned chroma-key panel to be able to 
view the panel the TV camera providing an output to a video processor 
connected to the camera said video processor calculating the camera 
viewpoint information, a graphics computer using the calculated viewpoint 



to render a pre-designed 3D scene, a chroma-keyer used to generate the 
composited foreground/background image, the apparatus further 
comprising an electronic image control means, said electronic image 
control means being connected to the video processor to enable a further 
5 pan, tilt or zoom operation to be actuated in addition to any pan/tilt or 
optical zoom operation already performed by the camera. 

Preferably the electronic image controller comprises joy stick 
control means for positioning a cursor on the video image provided by 
10 said TV camera said joy stick control means being operative to provide a 
pan/tilt/zoom action on the image to expand the region of the image 
proximate to said cursor position to enlarge a selected area of the image 
and to move "inside" the image provided by camera 12 by virtually 
panning, tilting and zooming in/out. 

15 

The same can be done on an image produced by the fixed camera 
in a chroma-key apparatus. 

The present invention also comprises a method of creating a 
20 television image comprising the steps of obtaining in real time a first video 
image of a defined area by operation of a video camera, displaying in real 
time said video image of said defined area, selecting a portion of the 
defined area expanding the portion of the defined area to fill the complete 
television image, displaying the expanded portion simultaneously with the 
25 display of said video image of said defined area and moving the expanded 
portion "inside" the said video image. 

Preferably the step of enhancing said video image of said portion 
is by texture mapping. 

30 
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Preferably the method is used in a virtual studio system, the defined 
area comprising at least a substantial portion of a patterned chroma-key 
background panel said substantial portion being sufficient to enable 
perspective transformation calculation of a virtual background to be 
5 included in said television picture, and in which said selected portion of 
the defined area does not include enough of said background panel to 
enable perspective transformation calculation of the virtual background. 

Embodiments of the present invention will now be described, by 
10 way of example with reference to the accompanying drawings, in which: 

Figure 1 shows a known virtual studio system incorporating a 
patterned chroma-key panel as described in copending PCT Application 
No. WO95/303I2 and is incorporated for explanation; 
15 Figure 2 shows a portion of the patterned chroma key panel of 

Figure 1 illustrating the present invention; 

Figure 3 shows diagrammatically apparatus according to the present 
invention; 

Figure 4 shows a video picture illustrating the technique of texture 
20 mapping as applied to the system of the present invention; 

Figure 5 shows schematically a remote controlled camera system 
incorporating the apparatus of the present invention; 

Figure 6 shows a first video picture illustrating an area to be 
enlarged according to the present invention; and 
25 Figure 7 shows the selected area of Figure 6 as enlarged in 

accordance with the present invention. 

With reference now to the drawings. Figure 1 shows a virtual 
studio 10 with a camera 12. A foreground object, in this case a presenter 
30 14, is positioned in the foreground area 16 in ft-ont of the camera 12. 
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In the preferred embodiment now described a background chroma- 
key panel 18 is provided which has a plurality of vertical and horizontal 
stripes 20,22 forming a unique pattern. The operation of this pattern in 
relation to a virtual studio is described in the above mentioned copending 
5 PCX applications. 

The camera 12 in this example is assumed to be operated manually 
by a cameraman (not shown). The camera can move (it can be shoulder 
mounted) in all directions within the area 16 and can also pan, tilt and 
10 zoom under manual control. 

As illustrated in Figure 2, when the camera zooms in to the face 
140 of the presenter 14 the number of lines 20,22 seen in the field of view 
of the camera becomes smaller. In this case the picture or image 24 as 
15 shown in Figure 2 contains only two horizontal lines 202,204 and two 
vertical lines 222,224. As described in the second copending PCT 
Application No. GB96/02227 the system is still able to work even though 
these lines are not continuous due to their disappearance behind face 140. 

20 However, if we now consider zooming in further to the area 

indicated by dotted line 240, it can be seen that no lines are visible 
because the face obscures them. This will also be true of other objects 
onto which a close zoom is required. 

25 The system will start to lose lock if it cannot see the lines 22,24 on 

att least portions of the lines and therefore the cameraman is constrained 
in close ups to ensure that the lines are visible in the picture 24. 

In the present invention as now described with reference to Figures 
30 3 to 7, this disadvantage is overcome by providing an electronic zoom 
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control separate from the camera optical zoom control. This zoom control 
may be operable within the control room of the studio or could be 
operable by the cameraman with a separate unit. The system operates as 
follows. 

5 

The cameraman zooms in to the picture as shown in Figure 2 and 
then ceases to zoom. The image to be transmitted is then shown in picture 
frame 24. 

10 For a closer zoom an electronic zoom controller is used to identify 

a smaller area, as illustrated with reference also to Figures 6 and 7. The 
smaller area may be identified by using a cross hair cursor 242 to centre 
onto the subject to be shown in greater detail. The zooming is then 
accomplished by use of a joystick or other suitable control to continue 

15 zooming onto the area 240 the image (of rocks and trees) being magnified 
as shown in Figure 7 to fill the complete image to be transmitted as shown 
in figure 7. 

The system does not ever lose lock because the image information 
20 input into the virtual studio circuitry still contains the image 24 as shown 
in Figure 2 or Figure 6 which still contains the image of the lines 202,204 
and 222,224. Thus the viewer sees a close up of the object (either rocks 
62,64 and tree 66 of Figure 6 or the face 140 of presenter 14 in Figure 1) 
and the camera 12 can continue to pan and tilt and move within the area 
25 16 to provide differing close up views of the objects. 

Because of the electronic zooming there will be a slight degradation 
in the quality of the picture because the image 240 has to be expanded to 
fill the whole picture screen 24. There is obviously a practical limit of 
30 acceptability in relation to the quality of the picture able to be transmitted 
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but theoretically there is no real limit. 

In practice, with a standard TV camera 12 a 50% magnification 
does not degrade the transmitted picture substantially. With a high 
5 definition camera and using high resolution background greater electronic 
zoom magnification may be practically acceptable. 

Techniques such as texture mapping as illustrated diagrammatically 
in Figure 4 may assist in allowing greater magnification or better quality 

10 pictures with the same magnification. Figure 4A illustrates within the 
rectangle 241 a target (original image) and Figure 4B the destination 
enlarged image. In Figure 4A area 244 is to be expanded to become area 
246 in Figure 4B. The average value of pixels in area 244 is used for 
area 246 to provide the magnified area. Thus the magnified image 

15 comprises the same pixel density as the previous image but the value of 
the pixels in a specified area is averaged. This is often not noticeable in 
any way to the viewer and therefore the electronic zoom does not 
practically degrade the picture. 

20 The apparatus and control circuitry 30 is shown diagrammatically 

in Figure 3. 

The camera 12 provides a foreground video image on line 122 to 
a perspective transformation unit 32. the perspective transformation unit 
25 32 extracts from the video image the perspective information derived from 
the background panel 18. The foreground video is then output on line 322 
to chroma-keyer 34 which may be, for example, an ULTIMATTE 
chroma-keyer in which foreground and background are combined to be 
output on line 342 and subsequently transmitted by transmitter 36. 

30 
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The background is provided by, for example, a graphics computer 
38 on Une 382. 

The background is generated by the graphics computer via 
5 rendering of a pre-designed graphical model at the calculated viewpoint. 

An electronic zoom controller 40 is connected to the graphics 
computer 38 and to the persjjective transformation unit 32 via respective 
outputs 402,404, the controller 40 having suitable controls such as a joy 
10 stick 42. 

Many monitors will normally be provided in the control booth to 
show the video router or studio director the image being transmitted or 
recorded. In this example three monitors are shown. The first monitor 

15 44 will show the scene as viewed by camera 12 with the background panel 
18. This enables the studio director to ensure that this image has 
sufficient of the panel 18 in the picture frame 24 to ensure adequate 
locking. It is possible to provide an electronic indication to the studio 
director when this is occurring to assist in the control function. If such 

20 loss of lock occurs then the camera must zoom back or move back. 

The joy stick 42 is then used by the studio director if he requires 
a closer zoom and the zooming is monitored on monitor 46 which shows 
the selected foreground image zoomed in as selected by the control 42 and 
25 electronic zoom controller 40. The studio director can also electronically 
alter the pan or tilt as well as and/or in addition to the zoom. 

The background is also modified by the controller 40 which 
controls graphics computer 38 to magnify the background or a virtual 
30 foreground object in the correct perspective and size to match the 
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electronically zoomed image. 



It is noted here that if the foreground object 140 completely fills the 
picture frame then there will be no requirement for a background image 
but virtual objects may be present which pass in front of the face. Also 
if only a minute amount of background is shown such as the area 242 in 
Figure 2, this will require to be added. 

The combined optically and electronically zoomed (and also 
panned/tilted) picture will be shown on the monitor 48 to enable the 
producer to see the effect of the electronic zooming/panning/tilting and in 
particular the picture quality. 

In Figure 5 a system is shown in which no cameraman is required. 
The camera 12' is in this case assumed to be static but has a mount with 
pan, tilt motors and a lens with zoom and focus motors controlled by a 
controller 500 which is operable by the video router or producer in the 
studio control room. 

The virtual studio is provided with 2D high resolution background 
502 which may comprise, for example, weather or other maps. 

The camera 12' can be zoomed into the background until only a 
small area is shown in the picture. To zoom in further would lose lock 
as explained above but by using the electronic zoom controller within the 
processing apparatus 30 further magnification of the image can be obtained 
to produce a more detailed picture. In this case, due to the use of a 2D 
high resolution background, no picture quality will be lost for any 
practical system unless of course the electronic zoom is carried to 
absurdity. 



The camera 12 can therefore be zoomed into a very small area 
which can then be further magnified. It can then be zoomed back 
smoothly without any loss of lock. 

5 The system therefore enables the studio controller to select 

extremely close up views in a virtual studio apparatus without cutting to 
a different camera or having to insert close ups later in an edited version. 
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CLAIMS 

1. A video control system suitable for a virtual studio apparatus 
including a video camera, said control system comprising electronic image 
control means for selecting a portion of a video image captured by said 

5 camera, said electronic image control means being operative to perform 
zoom/pan and or tilt operations on said selected portion of said video 
image. 

2. A video control system as clmmed in cleiim 1 wherein said video 
10 camera is a static camera. 

3. A video control system as claimed in claim 1 wherein said video 
camera is capable of movement in three planes x,y and z. 

15 4. A video control apparatus for a chroma-key production (blue box) 
system, said control system comprising electronic image control means for 
selecting a portion of a video image, said electronic control means being 
operative to perform zoom/pan and/or tilt operations on said selected 
portion of said video image. 

20 

5. A virtual studio apparatus comprising a patterned chroma-key 
panel, a TV camera situated in a foreground area in front of the patterned 
chroma-key panel to be able to view the panel; the TV camera providing 
an output to a video processor connected to the camera said video 

25 processor comprising a graphics computer to incorporate a real foreground 
object into a virtual background scene generated within the video 
processor, the apparatus further comprising an electronic image control 
means, said electronic image control means being connected to the video 
processor to enable a fijrther pan, tilt or zoom operation to be actuated in 

30 addition to any pan, tilt or optical zoom operation already performed by 
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the camera. 



6. A virtual studio apparatus as claimed in claim 5 in which the 
electronic zoom controller comprises joy stick control means for 
positioning a cursor on the video picture provided by said TV camera said 
joy stick control means being operative to provide a zoom action on the 
picture to expand the region of the picture proximate to said cursor 
position to enlarge a selected area of the picture. 

7. A method of creating a television picture comprising the steps of 
obtaining in real time a first video image of a defined area by operation 
of a video camera, 

displaying in real time said video image of said defined area, 
selecting a portion of the defined area, 

expanding the portion of the defined area to fill the complete 
television picture, 

displaying the expanded portion simultaneously with the display of 
said video image of said defined area. 

20 8. A method of creating a television picture as claimed in claim 7 
fiirthcr comprising the step of enhancing said video image of said portion 
by texture mapping. 

9. A method of creating a television picture as claimed in claim 7 in 
25 a virtual studio system in which the defined area comprises at least a 
substantial portion of a patterned chroma-key background panel said 
substantial portion being sufficient to enable perspective locking of a 
virtual background to be included in said television picture, and in which 
said portion of the defined area does not include enough of said 
30 background panel to enable perspective locking of the virtual background. 
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